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ABSTRACT ^ ^ ^ -..u- 

Qualitative measures were developed within a 
Principal-Teacher Interaction (PTI) study by the Research and 
Development Center for Teacher Education, Uni.versitjj of Texas at 
Austin, concerning the role of the principal and process of change as 
it occurs in school settings. Methodologies for data collection and 
analysis included quantitative measures of individual and group 
responses to change and qualitative dimensions concerning a sense of 
,the context, interactions, and social meanings underlying fhe . 
quantitative responses. These qualitative measures interpret and 
organize data to represent existing reality within a site and 
contribute to theory across sites. Qualitative me.thods can include 
coding. of change action interventions, listing of effects, levels of 
use of an innovation, and configuratioh and concerns data. 
Qualitative information can be^ focused by "using intervention mapping 
of planned action, antecedent mapping of how change evolved, crit.ical 
incident maps ,ai>d time lines, "and "reality checks" by -cross-s i te 
discussions involving "site expert" researchers and district , 
representatives. These« techniques provide a means for interpretation, 
cross verification, and validation of phenomena present in the data. 
(CM) ■ • . ' 
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Introduction , " * 

The topic of educational change has been the subject of'humerous studies 
over the past few years. ' During 1980-81, the CBAM Project, The. Research 
and Development Center for Teacher Education; University of Texas at^ftustin, 
engaged in a study of nine principals and their schools involved in the 
process of school change. This study, called the Principal-Teacher 
Intetraction study, focused on the role of the princ'ipal as the facilitator of, 
change within the school context. Specifically, the PTI study considered the 
role of the principal in terms of what principals do to fa<-ilitate change in 
their schools, how their style as a leader affects what they do, and what were 
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the effects ot^ their actions on teachers.- GBAM is -curfentTy involved in 
analysis of the data collected from the field sites. The long-range intent of. 
the study is to gain a better understanding of the role"''of^the' principal and 
jbhe process of change as it occurs in the school settfng. in order that some 
aids to promote facilitation of the process might be developed. ^ 

,' The field work in the schools pd subsequent analysis of .t^e data 
collected presented & unique set of problems to research staff. Approaches to 
the^e problems resulted in the development and use of combined methodologies 
for data collection and .data analysis. Preliminary wjDrk on data collection . 
was done in a pilot stu.dy conducted in 197'9-80 (Griffin, Goldstein & Hall, 
1981 ). Based on rpsul ts . f rom the pilot study, , a research strategy was 
designed that included both quantitative and qualitative measures. The 
quantitative dimensions were seen as a means to measure "responses to the 
change process as .they, occurred to individuals and the group in time. The 
qualitative dimensions allowed for 'a sense of the context, iri^eractions , ard 
social meanings underlying^ the quantitative responses. Each dimen-sion gave a, 
uniqJe perspective on the change process. In combination, they provided a 
meaxTS . for- interpretation, cross verification and validation of phenomena 
present 'in the data. 

It is the intent of this paper to discuss the qualitative measure's used 
for study analysis as they relate to the role of researchers as interpreters 
and experts for'.their sites. The paper also discusses- the ways' in which 
qualitative measures were used to interprel^and organize data such that it 
represents reality as it exi sts within a site and contributes to theory across 
s-ites. Specifically, the paper discus_ses analysis techniques such as site" 
mapping, ca\e studies, topical reports, critical incident maps, cross-site 
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discussions involving site researchers, and school- or district representatives, 
and the interpretation of "effects." Qualitative measures, as^they are being 
used in the study analysis, were designed to allow site researchers to focus 
the information they have as "site experts" in order to make comparisons 
between sites, draw conclusions, retain significant details; and maintain a. 

■ « " - : - ■ ■. /- . . 

richness of background 'upon "which to overlay other dimensions. The - 
methodology is, a developing -.methodology, based on experielices as th^y occurred 

• ■■ _ ^ 

in the field, and the pilot study, and aimed at providing a basis for continued 
work in approaches to change. . 

■ ■ Related Literature 

The us'e*of qualitative methodologies has gained increased popularity over 
the las,t decade. One rationale for this is as a reaction to an over-emphasis 
on qua,ntitatively based methods. Another reason migl:it be the special 
attributes of qualitative approaches and what th'ese approaches anight lend to» a 
research setting (Meyer, 1981; Lof land, 1971; Patton, 1980). 

QuaTitative approaches can most^ easily be described as the commonser>se 
knowle'dge that underlies any phenomenon. In an everyday context, these . 
approaches- are much the same as those means we as indiv.i duals use to make 
sense .of events that happen to us daily.' We observe, participate, categorize, 
analyze, and order the profusion of stimuli around us. For .research , this 
activity becomes a. process of selection from the social phenomena under 
consideration through which to develop understanding, hypotheses, and 
conclusions. Social phenomena in this case become the acts, activities, 
meanings, interactions, relationships, and settings— the forms they assume and 
the variation/ they display (Lofland, 1971). Ordering these phenomena is the 
role of trfe researcher who has the- responsibility of focusing the research 



gffort to the .questions qncifer study. The, end result of ordering may'be 
something . entirely different from initial efforts. .The use of qualitative 
methods in the- beginning allows for an understanding . of situations and 
participants in^ situations such that descri pti ve^categories may be d'erived. ^ 
Meyers (1981) suggests that these descriptive categories allow the researcher 
to explore and describe events as they are becoming understood io allow for 
later quantification, formulation of variables, or the/ development of other 
more sharply defined categories.. In this sense,\the qualitative base allows 
for the emergence of categories from the data^nd the "beginning of an^ analytic 
.structure in looking for interrelationships across categories^,(Lofland, 1971). 
The develofment of loose descriptive categories, concepts, or theories tha-t 
are. grounded in the data is referred to by Fil stead ' (1979,: p. 36) as "first 
order": concepts. These,, in turn, are essential in the process^jSf developing 
second order concepts, or' those that emerge from attempting ' to explain the 
phenomena. ' • ' , . ' ' 

Qualitative data according tcrMeyers (1981), Jick (1979), McCutcheon ' 
(1981), Glazer (1972), and others haA/e the added advantage of providing the' 
necessary background for' testing the validity of other types^of variables, 
such as quantitative methods.. In fact, he says: J ^ 

^ Qualitative and quantitative studies are not so much • r 
. 'complementary or convergent as inseparable; One is 
necessarily related to tha.ather.,^ Quantitative methods 
cannot be fruitful unless qualitative data are used to 
inform the interpretation of the desigxis and^varriables 
' ^ ' (1981, p. 162). . " - ' • 

M,any researchers have advocated the use 'of qualitative and quantitative 

^ ' <f , 

methodologies together, to .provide additional insights that neither,, alone 

could supply." This use of multiple methods to increase focus cn the 'same 

phenomena has been 'calleeii/'trianaulation'i (Jick, * 1979; Denzin, 1978).. 

Triangulation 'has been u^'ed largely as a vehicl e' for cross validation a's in 



the case where two or mgje distinct methods are found to be tonvergent and 
yield comparab.le data (Jick, 1979). . A special feature of qua! itati ve_ methodjs 
i*n trj^ngulation is in eliciting a breadth of data of illuminating elements 0^ 
contex1;,Jhat allow deeper dimensions to emerge.- Non-ponvergence of data in r 
triangulation is al so important in that it re'quiresi.a reassessment of 
methodology or database to establish the basis *for differences. This in ^ 
itself can add richness to the study. ' 

• : The process by which descriptive categories are refined and reformulated 

^'h terms of the goals of the research is called qualitative analysis . Lofland 

(1971,' p. 118) suggests that a temporal overlapping of/analytic and data 

collection activities ' is desirable in that there/fs' a developmental tie v 

between early data. cTlTect ion and later result^ 'He says, "The final stage of 

analysis (after 'data collection has ceased)^becomes the period of bringing. 

final order to previously developed ideas; Analysis begins with ifeose 

assemblies in terms of topical areas and with a consideration of the overall 

structure into which the data' is falling," (Lofland, 1971, p. 118). In 

analysis the qualitative paradigm allows for a dynamic i-nterchange between 

theorl, concep'ts, an^. data with constant feedback and modifications of theory 

and concepts based on the data collected. Filsteaxi (1979, ^p. 38), Jri a • 

simil'ar, vein to Lofland's interrelated topical areas'and structure, calls this 

emerging theory the "explanation frameworlj;" that gives direction as to where 

additional problem solving or information is needed. 

' fhe act of developing an ."expl anation framework" or analysis may involve 

different methods.. Meyer (1981) describes Blumer's points of "exploratory 

inquiry"' which include methods of inquiry such as interviews, group- 

discussi'Cfn,; participant observation, and examination of records to construct 

an accural picture of the social world under study. This isjfollowed by 

I 



"inspection" which is the gradual ^ecifying" of boundaries of a category by 

careful attention to |ncidents\within each category (Meyer j .198i , p. 188). ^ 

» . ■> • ° -iv. 

Glazer's (1972) suggestions, for analytic approaches include .something he 
^ .. ' 

calls a "constant comparative method" which includes: a)^-comparing incidents _ 

in each category; b) integrating categories and other properties; c) deriving 

hypotheses from <;ategories and context;'and d) developing theorry by relating 

categories to one 'another. Glaser al so .suggests the process of analytic 

induction which defines the phenomena, formulates a hypothetical explanation, 

and then applies that explanation to one test case. Depending on -Results of 

the test case, further testing is donejn more cases, feviewing the data to 

see what hypotheses are suggested (Meyers, 1981, p,. 189). . ' 

If analysis is the fi;<amina]:idn of the data collected with con^si deration 

given to the major elements and their interrelationships, interpretation 

reflects the meaning found in that data or examination,, as' wel'l "as the 

processes involved in making that meani^ng. McCutchetn (1981, p. 6) se'BS the 

process of interpretation as 'involving three steps: 1) the forming of 

, patterns accounting for the affil iation^of separate phenomena to one another; . 

{) the interpretation of the '^social meaning of , events through, "thick 

description," or qualitative background; and ^3) the relating, of the 

'» ' ■ 

particulars of the setting to external considejations, such as theories. An 
interpretive format would then, involve both considerations internal to the 
site, as in the data collected, and external as might be/shown in research 

t * 

'questions or theories applied^to analysis. Her critt^ria for judging the 
accuracy of interpretive cbnclusions include „the logic inherent in the 
interpretation process and siiff iciency of' evidence based on multipj-e data 
collection periods or visits to the site. It is important that the line- of ■ • 

♦ 

evidence hold together structurally. Also, she sugggsts;;that accuracy may be 



measured through agreement with o1;he'r forms of information as in 
triangulation, through significance of results, through^ universality and 
general izafcility of results, as well as from a solid transaction between the 
researcher and the phenomena. McCutcheon states, "Interpretation has been 
discussed as a transaction between the researcher's knowledge and the 
observations being made. The interpretation process places the researcher in 

f 

\ 

an active role/in the construction of meaning," (1981, p. 9). 

The role of the ^ researcher as the. expl-orer and interpret-^r. hf the 
phenomena i's a significant one. By participating in the culture stud^i^, t*He 
researcher is socialized into that culture "which gives him/her special 
knowledge of -'events in* the cul ture— knowledge that may be applied to all ^ 
aspects of a research strategy— from data colection to early categorization 

•» * * 

and formulation of hypotheses, to data anlysis and interpretation. Further, 
the researcher is an important variable in linking methods together, as in the 
use of convergent methbds, on the basis of his'/her qualitative Isnowlejige. 
Meyers (1981, d> 190) says that qualitative data and analysis function as the 
gl.ue thaU cements the inteVprelation of. multimethod results. 

Finally, i-nter-researcher exchanges and discussion have been shown to 
have value in allowing for definition o'f researcher bias in debriefing or 
other discussions, in developing topical categories-, a-nd in looking for 
interrelationships within the data (Meyer, 1981; McCutcheon, 1981; Filstead," 
1979; Reic^ardt & Cook, 1979). The qualitative paradigm stresses a, negotiated 
view of the social order, negotiated through .a dynamic interchange^ between 
researchers, ' information, theory, and resolutions as a part of the analytic 
process. _ ^ • 



\ The Principal'^Teacher Interaction Study: 

Background on Data Collection 

The Principal-Teacher Inter'action (PTI) study focused on the principal's 
role as the. manager o'f change in his/her school. Three sets of schools, nine 
schools in total, were chosen, each at a different stage in the implementation 
of curriculum innovations. While the focus of the study was on the principal 
as the manager of change, questions were directed to problems specific to the 
stage of implementation aind the schools involved. "One set of these, schoals 
was in. first year implementation, another in second, and another set was in 
the third year of ' imp! ementation. Within each set; school^ were selected 
based on the leadership style of their principals, as judged by district 
administrators. These leadership styles were.. hypothesized to be related to 
their concerns (Hall, Rutherford & Griiffin, 1982). 

The methodology designed for data collection at thfese sites was based on 
preliminary work with schools in Jefferson County, Colorado in 1979 (Hall, 
Hord & GrKfin,'1980), the "pilot study (Griffin, Goldstein & Hall, 1981), and 
an extensive review of. the literature on the role of the principal. The. pilot 
study^ in particular, provided the vehicle to test mpst of the methods used in 
.the PTI field stydy. Based on results from the pilot study, a^mbination of 
procedures for documenting principals' actions with respe-^t to change, cal,le8 ^ 
interventions , v^ere developed (Rutherford, 1980; Hord, 1981). The procedures 
used in the PTI study included logs written by principals, bi -weekly phone • 
calls to principals .by R&D Center staf^, and on-site interviews with school 
administrators and teachers. The interventions collected by these methods 
were then recorded and coded by project staff (Hall, Hord & Zigarmi, 1980). 

During the field year 1980-81, CBAM staff members were each assigned one 
principal and one school to work with in the PTI study. Each group of three v 
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res.earch^rs assigned to a district would travel together, .wor^ with distri^Ct 
officials as a group, and compare notes on similarities and differences in 
their schools within, the district. In Tatar analysis discussions these 

i 

district research groups became the foundation for comparisdns between sites, 
for^ establishing reliability of inter-site data base, and: for developing 
theory. R'e^searcher Ve^ponsibil ity was dual: on the one/hand, they were*** 
engaged in data collection and interactions with individuals at the school 
site; on the Jther, they Were recording and- coding behaviors and interventions 
as well as working for reliability within the coding scheme (Hord & Hall, 
1982,). Researchers were also responsible for maintaining notes on context and 
interactions as they observed. . . • 

' The time 11,t^ for ciata colljection at each s^te was as follows (see Figure 
1): preliminary interviews with teachers,' district personnel, ^and the 
principal and assistant principal^ were conducted in the spring of 1980. 
Interviews and ,Stages of Concern measures were given, to teachers in the fall 
of 1980, winter of i'981 , and> spring of 1981. Interviews conducted with 
teachers were' focused to include Levels of Use (LoU), Innovation^ 
Configurations (ICj, and intervention data. These procedures measured changes 
in concerns and actions with respect to the innovation. On-site interviews 
were conducted with principals du/ing the same time frames... Principals were 
telephoned bf-weekly t'o record ' the interventions they had logged. These - 
telephone conversations had the added benefit of maintaining a level of 
personal rapport between principlil and researcher, and gave more information 
about school background 'and context. CBAM staff also designed a context 



^Many schools had both a principal and assistant principal. Ror the sake 
of simplicity, any reference to th§ principal includes potential of a 
assistant, principal . __In schools that had both a principal and an assistant 
principal, data were- coTi-ected from both. 



Figure 1 

Measures Used in CB/^M Principal -Teacher 
Interaction (PTI) Study ^ • 



Indivi-dual Teachers 

Stages of .Concern (SoC) 
Levels of Use (LoU) 
Innervation Configuration- (IC) 
^ Interventions 
Interviews 



\ 



Measures taken 
May '80, Oct; '80, May 'SI 

May :8Q, Oct. 'SO . . 
'Jan. !81, May '81 



Principal and Resource'Teacher 

Change Facilitator Stages of Concern 
Questionnaire (CFSoCQ). 

Interventions 

Interviews 

Bi-weekly pnone interviews (principal only) 



\ 

May '80, Oct. '8i 
Jan. ''81, May '81 



■ Sept. '80. - Ji.rie '81 



Group 

School Ecology Survey (SES) 

Observation 

Situation Surve> 



Jan. '81 .V 
May '^0,. Oct. '80, 
^Jan. "81, May '^1 



District 

Interviews 
Situation Survey 



.May '80 - May '81 
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measure,, called a School Ecology Survey (SES) which was given to teachers in 
the winter of 1981 (Griffin & Hall, 1982), 

At the end of the data collection period researchers had information on 
•individual teachers across /the school year, information about the school 
environment or situation, / sense of social interactions with'in the school, a 
sense of teacher leadership and role, indications of the pr4ncipal's social 

interactions network anci leadership within the school, principal interventions 

. . _ _ • 

as d'escribed by the .principal , interventions and informatiiOn from resource 

teachers and teachers, as-^well as a sense of the school ' s relation to the 
district. All of this information v;as considered important in the data base. 
Most of the information was contained on tape, in notes, formal and informal, 
or inrluded as a parf of intervention coding. Despite a lack of a lengthy 
stay within the school, which would be traditional to n.ore ethnographic 
methods, the combination of quantitative measures— SoC , LoU, IC SES, .and 
interventions— did allow researchers to get a. sense of change as it -occurred 
for both the individual and the school, as well as a background on which to 
place that information. It was at this point that site and cross-site 
analysis could begin. 

I , ' 

[ Qualitative Approaches to Data Analysis 

.1 . • 

Following the year of data collection, the CBAM research staff was faced 
with a vast amount of descriptive data on interventions collected from . 
principals, assistant principals, district change facilitators and tfeachers. 
In .addition, each, staff member had collected field notes on conjlext, 
leadership style, and principal ' s philosophy, and more structured descriptive 
information on schSol background and environment. Quantitative data, 
measuring effects of the implementation effort on teachers' concerns about 

11 



(Stages of Concern)- and use of (Levels of Use, Innovation Configurations) the 
innovation,^ had also been collected. These data soyrces became the basis of^ 
data analysts. . • ' 

Data analysis involved both quantitative and qualitative approaches. 
Qugntitative analysis ' of the intervention- data required coding each 
intervention using two analytic frameworks' devisad for this purpose: The 
Taxonomy of Interventions ^Hall, Zigarmi & Hord, 1979) and The Anatomy of 
Interventions (Hord, Hall, Zigarmi^, 1980). Appropriate quantitative- 
analytical procedures were performed (Hord, 1982). The same data were also 
analyzed in more qualitative ways. The interventions were "mapped out'* in 
different ways to uncover relationships and events 'between individual 
interventions and between interventions and their effects on teachers. 
Effects of interventions on teachers were also analyzed both quantitatively 

4 

and qualitatively (Huling, Hall & Hord, 1982). 

A number of qualitative strategies were adopted for analyzing the entire 
data set-^interventions, effects, leadership, context, and background data--in 
'order to develop a total picture of the implem^irtation efforts,*" to answer the 
the study's major research questions and to generate hypotheses for further 
analysis. Tv/o techniques for formulating case studies were developed""the 
case study interview and the case study topic report. Different approaches to 
cross-site analysis ware also employed, including topical discussions between 
researchers at the "various sites, as well as discussions between researchers 
and study-site district change facil i-tators. As the individual site 
researcher was the "site expert" on the data s/he collected, these procedures 
became the means to utilize and focus their information base. Descriptions of 
these qualitative approaches to analyzing the study data follow: 
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Intervention Mapping ^ ' ' ^ 

Mapping by intprvention level. Part of the CBAM project's 
conceptualization of interventions is the idea of Levels of Interventions. 
Definitions have been developed which describe a hierarchy of levels of 

a 

actions sponsored or planned by change facilitators to'implement a new program 
(Hord, 1980). The highest ilevel is the Game Plan, which' is the overall or 
generalized plan of action for the change effort. The Game Plan is then • 
broken down into its- functional components, called Game P,1an Components^. Game 
Plan Components are specified by Strategies which are then further broken' 
down, or operationalized, into TactiQtS;f^M Incidents.. •' - ' 

Membsrs of the CBAM project have devel^)ped a technique for building°this 
hierarchical picture of a change effort, referred to as Intervention Mapping 
(Zigarmi, Goldstein & Rutherford, 1978). Interventions, written up as short 
statements of actions or events, are arranged hierarchically into functional 
groups. Incidents are placed be,neath the tactics" which they operational ize , 
ta'Ctics under the. appropriate strategy, and strategies un^er game plan 
components. Figure 2 provides an example of one small part of an intervention 
map. Note that the interventions are also mapped horizontally by date of 
occurrence. 

Intervention maps are developed in both a top-down a'nd bottom-up fashion. 
Incidents can be grouped^ into tactics, which are then built up into 
strategies, and so on; or-, one can start with a known .strategy and locate the 
tacticjs and incidents which fall beneath it. ^ Most often, maps are built up 
from incidents or tactics because higher level interventions, e.g-. , strategies 
and game plan components, often are not pre-planned or verbalized by the 
subject from whom the intervention data are obtained. 
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' Figure , . 

V 

School A Intervention Map Sample Page 



Game Pfan Component 2 : Training in Use of Composition Program 

V strategy ^2.1 (158). P^rincipal , conducts a series of staff meetings to introduce, 
clarify, and train for use in' different aspects/of the composition program./ 

. Tactic 2.1.1 (1&9). Principal conducts meeting on us.e of sourcebook. 

/ Incid ents : '56, 37, 35 • - ^ , / ^ ■ 

♦ - — — . " ' . *- " ■ 

.1 Tactic 2.1.2 (64) . Principal conducts^ meeting to- discuss "^scope and sequence and 
ito .introduce rubrics. ^ ^ 

' Incidents : 65, 66 ^ 

Tactic g.-1.3 (160). Principal conducts 'meetings on 

Incidents: 80, 81, 79, 84 . ^ 




Game Plan" Component 3 : Consultation and Reinforcement. _ . 

Principal emphasizes need to improve, in areas of language arts and composition 
by reviewing we'a^knesses as shown in school testing. He provides new information 
on teaching "composition to hejp resolve weak areas and uses .Resource Teacher,. 
Amy Bauman and individual teachers .to v^ork with other teachers in encouraging 
use of new material. . 

strategy 3.1 (135)T->Principa1 uses CAP/CAT sc&res to reinforce need to 

improve in areas of language arts, including composition. ^ . o 

<? 

Tactic 3.1.2 - (23) . Principal includes langauge arts and composition in 
some teachers' objectives on the basis of CAP/CAT scores. 



Incidents : 27, 26, 25., 24 



Tactic 3.1.3 (145)-. Principal meets with upper grade teachers to discus^ 

methods and goals for' composition and language arts on the 1981 CAT tests. 

- ,v ' J 

Incidents' : 85, 86, 28, 131, 180, 75. ' 



Cj... 

Strategy 3.2 (155). Principal uses key teachers to dissemina^te infq/mation and 
to develop staff support for composition program. ^ 

Tactic 3.2.1 (157). PriQcipal appoints teacher .4397 to be in charge of 
School contribution to Di strict Magazine REFLECTIONS. 

* *■ 

Inc idents : '44, 62, 76 ,-100 

.. " '^io ^ 



Mapping antecedent interventions ^ Interventions may be mappeci in yet^ 
another way--through buildffng a kind of path chart. To do this, one* would 
begin with some sign^ificant or central intervention and attempt to trace and 
display: . 1) what subse^quent interventions, it led to,, and 2) what 
interventions led up to it. Figure 3,,is an illustration of,, how an antecedent 
intervention, map might look^ 

InterventMn mapping has a number of benefits as an analytica*! approach. 
Intervention lev.el mapping builds a conc^tual picture of the whole change 
effort by showing how interventions are related to one another by function and 
by time^ they occurred. Antecedent intervention mapping shdws how one 

intervention leads to another. It captures a picture of the change process on 

\ * . ■■ - ^ 

a"" more microcosmic level than intervention level mapping. Both kinds of 

mapping show, visyally, how .the change effort unfolds over time. This, picture 
J. ■ . ^ . 

• t) 

can be related to. quantitative measures of effects on teachers taken at 
various points in time. For example,, a profile showing changes in concerns of 
teachers over time can be locked at with reference to an intervention map in,^ 
an attempt to re.late specific interventions or groups of i,nterventions to- 
.increases or decreases in concerns intensity. 

' Crit jcal Incident Map . Along with the Intervention map for each. site, 
6ite researchers were' called upon to develop what was • called. :the "critical 
incident map and time' line" of events as they occurred at theirJndiv^iduaT 
sites. The maj.or emphasis of this map was to org^ize events in time, showing 
particular incidents that seemed to have impact on the change effort. Some of 
these incidents were directly .related to the change process;, others were more 
strictly related to the social structure of the school and only indirectly 
served as catalysts for adtion toward implementation. Researchers determined 
th-o^e events^ 1 isted on the Critical Ii^ident map through observation,' 
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Figure 3 , 
Anjtecedent l!nte'rven.tion Map 



■ \ 




interviews with teachers and the principal, and from overlaps in intervention 
. data as it was papped. Drawijng up, the Critical Incident map and time line * 
proved a valuabi e. tool when done in^conjunction with the interv^lition analysis' 

» and' mapping in determining a sense of order to events as they contributed to,^ 

' ..I 

„ or bedame a part of, tactics and strategies. '^A" very general critical incident 
map^ often preceded formalization of the intervention map. ^ * • 

One example of the critical incident map^lfc^shown in Figure 4 . This 
figure represents events as they occurred in the first year of implementation 
' of a writing composition program. Events that were -critical to the 
\ development of b school -based' proagram are marked with an asterisk.^ Events 
that had a. strong catalytic effect on the principal ' s game plan for 
implementation, or in. determining the nied for a more focused ^school -wide' 
^ ' \ pVogram'are boxed. Relationships between events can be seen as both' temporal 
. and'^ d^^^^^ an . exchange of .information, '^n this scliool , teacher confusip^ 
. about thq^lprogram, discussion of teajcher concerns, and new information about 
the program from the resource teacher all had significant effect on .program 
developme 

The CriticalXincident map allows the researcher to see significant 

relationships between clusters of incidents in a simpler format than tjpe 

larger Intervention'map. Events as €hey are listedrin the Critical Incident 

map allow' for a schematic o{ the process as it develops in time. Within this 

si^ematic the contextual influences and roles ^^jsignif icant individuals can 
• . ^ ' • * ^ * . 

be seen as they relaie to the change .^ocess. In' this sense, the Critical 

Irffc^dent Map has an additional value as an ethnographiag^itool , or a shorthand, 

descriptive way to record th^ process and events involved 1n change at the 

iijdividw^l site. 
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Workshop for 
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Hawthorh -School Critical 
Incident Chart & Time Line 
(Abbreviated from Case 
Study Map) 
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Dec, 
1980 



J'an. 
1981 



12/1; 12/15 
Principal & 
resource tea- 
cher meet to 
plan composi- 
tion committee 
meetings 

12/1 

Composition 
discussed at 
staff meeting 



11/10 ^ 
Principal & 
resource tea- 
cher meet with 
committee (in- . 
eludes 6th 
grade teacher) 

11/25 

Resource tea- 
cher meets 
'with committee 
in I sections- 
upper and lower 
grades * 



12/10- 

Resource teacher 
meets with com-__ 
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to review scope 
k sequence 

12/18 

Resource teacher 
meets with com- 
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to plan* 



Principal meets ^ 
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tea^cherS to encour- 
'^"ge' work on compo- 
sition-related 
activities 

11/25 on - 
Corrmittee teachers 
meet with others 
at grade level 
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Resource teacher 
dis-tributes scope 
& sequence to all 
teachers 
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conlfext 
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cher introduces 
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with other staff 
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Staff meeting 
to clarify 
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Staff meeting 
to check use 
of Rubrics 
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Resource tea- 
xher works 
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on Rubncs 



Principal 

Includes Rubrics In 
school *newsl etter 
to parents; collects 
essays for publica- 
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Feb. 
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May 
1981 



12/1; 12/15 
Principal i 
resource tea- 
cher meet to 
plan ccmpos4- 
tion cornmittee 
meetings 

12/1 

Composition 
discussed at 
staff .meeting 



12/ljP 

Resource teacher 
meets with com- , 
mi t tee sectiq^ns 
' to review scope 
& sequence ... 

12/18 

Resource teacher 
meets with com- 
mittee sections 
to pl.an* 



12/16 

Resource teacher 
distributes scope 
i s'fequence to all 
teachers 



1/12-15* 
CBAM inter- 
■ view re. school 
context 

Prihcipa-l regf^ 
sters .concern 
about context 
u comparative 
impl-ementation 
at other., 
school s 



1/7* 

Staff meeting 
Resource tea- 
cher introduces 
Rubrics * 



1/6 

Resource teacher 
mefets wi tli com- 
mittee to plan 
staff neetiflg 



1/15-2/24 
Principal 
monitors use of 
Rubrics 



2/8-15 
Sixth grade 
competency 
ttests given: 
incl uded 
composition 



5'?/ 24 

"^''^Principal reminds 
cormii^tee to work 
with !6ther staff 
on f^dbrics 



3/3* 

Staff meeting 
to clarify 
Rubrics » 

3/24* 

Staff meeting, 
to check use ' 
of Rubrics 



3/1-30 

^source tea- 
cher works 
with teachers 
jiidi VI dually 
Wi Rubrics 



ppihcipal 

includes Rubrics in 
*schoO"l newsl etter 
to parents; collectt 
essays for publica- 
tion , 



June 
1981 



' 20- 2^- 
School-wide 
achievement 
tests 



Principal i 
resource tea- 
cher meet to 
review compo- 
sition ^ 



Principal, 
meets with 
individual 
^ teachers on 
'.composition 
• objectives 



5/12-14 
CBAM 

interviews 



5/14* s 
Principal , 
resource tea- 
cher^ commit- 
tee meet to 
plan for 
closure & 
^ 1981-82 year 



5/18 5/14-15 

Resource teacher District resource 

.distributes memo teacher plans for 

oh further plans 1981-82 co^positj^or 
re. composition 
to teachers 



^6/1 

Teachers get 
■ achievement 

test scores 

6/12 

School ends 



6/16 

Principal & 
resource tea- 
cher meet re. 
composition 
for 1981-82 
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Effects ^Qf Inteicventions . . • 

In completing codification of interventions ma^de in tbe court^e of cha^e 

.^t the individual school, researchers were called upon to list effects as they 

saw them result frdm the intervention (See Figure 5, Coding form). The source 

for deriving .effect was multiple, depending upon the nature of the 

intervention and its possible impact. * On the incident level, effects were 

usually qualitative, based on responses or results as perceived by the 

researcher or as mentioned by the school staff. In some cases, no immediate, 

effect could be seen. . 

*♦ ■ 

■ f ' - ' . 

One example of an intervention and effect on the incident level^is as 

follows: 

: . ^ « 

Intervention : the Resource teacher explained the writing program 

evaluation system to all the staff at a staff meeting'. 

. Effect : 1. T!">2 fetaff practice'#i|se of the evaluation system in^ 

the staff meeeting, in order to develop some understSindti^g 
and consistency before use in^ the classft)om. 

.2. The staff was notified that a follow-up meeting would ^ 
take place in. a few weeks to resolve any^ problems and 
answer questions, ^ ■ ^ . 

The source for information on this effect was the .report of the resource 

teacher and other teachers. As many incidents and incident effects are 

.linked), the secondary effect^ became a later incident, i.e., the follow-up 

meeting, leading to a tactic: the principal plans and conducts meetings'to 

develop use of the evaluation system. _ y 

On the 'tactic and strategy level, effects of interventions may show up in 

both qualitative and quantitative sources. For instance, the strategy^ related 

to the tac1;ic and incident mentioned above— principal eonducts a series of 

staff meetings to introduce, clarify, and train for use.in different aspects 

of the writing program--had the effect of completi'ng that training over the 



Figure 5 
Intervention Coding Form 



INTERV^ITION CODING FORM 
I . ' I(!ientifiers 



30 / 



Site 



Intervention ^ 



Date of Interview. 09 / 16 / 80 Person Interviewed .P30 
Linking Sponsored Intervention ^ Linking Theme # Jt_ 

Antecedent Intervention #'s 184 ' 3_ 

Intervievyer 'sB Coder SB 



Transcript Page 



Line 



ll. Brief Statement of intervention: District sponsors inservice on Composition, 



\ 



^ 1 ^~ . '■ . \ 

^ schools in the district'.. 




1 


• 


• 


....... , , . 

III. A. Intervention Level (circle one) ^ i 

1) Game Plan Component ; 2) Strategy ^T)^Tactic ^/\) 


Incident 5) Theme -6) Policy 


B.l. -Coding for Incident or Tactic Level (give code 

SublevQl Source Target Function Medium Flow 
' 11. 8A ' 4C, 10 IE, 2A 1 '1 


v' , 

Location 
2 


Time Duration. . 

1 

mi days /mo 
^ Dates 


' ■ ■ ' 

2 ^Coding for Strategy Level (give code //) 
Source Target Function 



3. Coding for Game Plan Componeivt\ci rcle one) 
(T), 2 3 4 5 6 . 

4.. Further Description of Theme or Policy 

Teachers report inservice not very exciting or relevant. Many don't 



Start 09 / 13 /8p 



mo day yr' 

End 09 / 13' /80 
mo day * yr 



^• ^en remeKiber it. It did not inspire them to use Sourcebook. 




IV, Diagnosis\(c rrcle one and describe) 
. SoC 0 



LoU 



0 

1 



1 2 


3 


4 5 


6 .1 






ni , 


IVA fVB 


V yV-r 




2 3 


4 


5 6 


7 8 9 10 11 12 
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CONT'D 
20a ^ 



/ 



2 4 



-over- 



EFPECT5 . ^ . I ; 

1/ Teachers received Sourcebook * 

2. ^eacher^ got a brief overview of contents, including: 

1. prewriting 4/ domains (See memo on June 30, 1980 froqi district) 

2. composing skills .5. evaluation techniques 

3. editing ' 6. management techniques 

3. IC'(10/.8O) shows £hat teachers, dtd not see, this workshop as a beginning of inno- 
vation--possib1y as a supplement, but do not remember an emphasis^on domains tfr 
evaluation techniques. Mast teachers remember ^the workshop' focusing on -things 
that were similiar to what they were already doing and they perceived no need 

•> to change their present practice. , . / t ' - 



OTHER INFORMATION '(Use quotes i^/henever'Dossible) 
VI. INDICATORS OF lEADERSHIP STYLE 



Vn. INDICATORS OF hHILOSOPHY/BELI EFS 

Principa> said he saw Sourcebook as means to help teachers with their language 

arts program--in particular as complement to scope and sequence, not as program 
in itself. ^ 



VIII. OTHER COMMENTS 



ERLC 



Principal is unclear as to what Composition Program is.. Sees^it.as a part' of 
language arts program-, but not an emphasis in itself. 



ref: ^ . .» ' . 

Herd, S. M. & Hall, G. E. '^Procedures for QuantatiVe Analysis, of Change 

Facil itator. Interventions. " Paper presented at th.e annual meeting, of 

the American Educat-f^naTrR^search Association, New' York, 1982. . ^ 

V / 20b. 



course oT\the ichool year so that teachers were using the writiag system in 
their clasWooms. The source for measurement and verification of that effect 
was the prlincipal , resource t'etcher, and teacher report, but also included 
•changes inlteachers' SoC, LoU-, ar>* Innovation Configuration data as recorded 
during Jatal collection periods by study staff. Teachers moved from high 
^x:oncerns ab(|ti-t.use and non-use of parts of the writing program, especially • 
evaluation iechniques, to use as a' result of training. This strategy 
. corvtained other |, 
tactics and |nciden.ts, Ijke those shown in the example, all with effects as 
they related tp those incidents that- in turn built to more 'major dimensions. 

The revdrse side of the coding sheets is designed for the recording of 
effects and includes the "Indicators of Leadership Stylg (principal),;,' 
"Indicators cjf Philosophy/Beliefs '(principal)," and "Other Comments." These 
sections werb intended to allow researchers to reference background 

•i'nfomiationV Isources., and relationships betweei interventions to the specific 

■■ I , ' ' • 

intervention jand effect. As "one inci'dertt/ef feet built to another, they had _ 

I ■ ' " 

the added advantage of allowing the researcher to hypothesize on paper as to 

the direction they saw the school , change effort, or princi'pal 's leaderji^hip 

taking. These hypotheses^, ususally a part of "Other Comments," became 

valuable in developing the Intervention map, the case study, and as basis 

« 

for interpretive discussions between staff. ■ * ■'•5, • 

Effects as an interpretive device had a special value in that it called 
upon the researcher to consider the background dnd attitude behind each level 
of intervention and to relate different parts bf the research strategy *to one 
another as sources for effects information. Each listing of effects and other 
categories unde'r effects presents evidence building to an overall plan of ^ 
action for each school. Further use of effects and the information contained 

I 

*. 
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in effects are still in the process of development as it is to be applied to 
analysis. ' 

Background For Interventions and Study Arialysis 

School Case Study Interview . The School Case Study Interview was 
developed by CBAM study staff specifically as a means to tap the depth of 
information known to researchers as "site experts,'* as it applied to the major 
questions of the study. At the time of development of this approach, research 
staff were dealing with many different facets of . final izing data coTTection 
and organizing" in-formatiort for ini'tial steps of analysis. As a practical 
attempt to make for^more spontaneity, better recall , reduction of work load 
and writer's block, CBAM staff decided to ^apply some of the same research 
strategies in-house, as hati b'een applied on site. The final product was a,, 
focused interview designed to be given by staff members to each other, to be 
tape recorded and transcribed to text, allowing sections to be expanded with 

more explicit details or drawn from in the course of further discussions. 

> " . . .. . 

The Case Study Interview included questions under five ma-j^r headings: 

1) background, context, and gejneral infonnation on^ or impression^" of , the 

school , principal , and staff.; 2) the nature of the innovation and its role in 

the school; 3) the yeaj's interventions; 4) interpretation in terms of 

research questions; and 5) synopsis of findings,. Under each heading sffort- 

answer questions were developed to narrow discussion or descriptions. The 

following is a^ sample of the questions under the first headirtg: 

What are your general impressions of your school? 
^ What do you think of first when someone says school? 

How would you describe the* teachers as a group? How many males and 
females? How long have most teachers been at the^school? Do they 
seem to get along well? How well do they work together? 

How.W£)uld you characterize the principal's interactions with people? 

'Z'Z 



What would' you sa'y about the principal as a leader? What would he/she 
say about himself/herse'lf? 

Before beginning the interview, staff members were asked to 'review their 
field notes, the situation survey, their Critical Incident map or time line on 
events at the' school , and any SoC, LoU, Innovation Configurations, or 
intervention data they thought important to the discussion. Interviews were 
limited to about sixty minutes. Researchers were asked to speak spontaneously 
as much as possible without reference to background materials once they had 
reviewed it for tjnemselves. Preliminary testing had found that reference to 
detailed background material, ?uch as was contained in the situation survey or 
quantitiative data,, slowed down the interview significantly and reduced 
spontaneous recall. Much of this type of information was added upon 
transcription. . 

A shorter, but similar format interview was used. to develop a case study 
on each district. , • . , 

The Case Study interview allowed researchers to make concrete the 
information known to them through their involvement at the site. The text 
developed through transcription became a source for discussion material, 
topical reports, and reference. The informal interview had an additional 
"advantage of .allowing researchers to express their feelings, ideas, biases, or 
hypotheses about the site through informal interaction with another staff 
member. Researchers reported that this ip itself gave a measure of personal 
satisfaction and reduced anxiety as to how their personal knowledge could be 
handled. 

Case Studies: Topic Reports . In addition to the case study interviews, 
which provide 'a gestalt of the implementation effort-^-and context at each study 
site, the'CBAM project will prepare short reports focusing on specific issues 
or topics. As the quantitative and qualitative analyses of the PTI study data 



proceed, questions often emerge which require descriptive information and/or 
researcher impressions on each. school and principal. Examples of the kinds of 
questions that could, and have, emerged are: What are some indicators of 
principa.1 leadership style? What elements of the school context seem to 
affect implementation of the innovation? How have principal concerns .about 
'implementing the innovation changed over time? 

With the particular question or topic in mind, each researcher calls upon 
his/her ethnographic notes, intervention and context data, p'ersonal 
impressions, and text from the Case Study interview, and prepares a short 
written report on the topic. The research team member who posed the question 
then collects these data and studies them for cross-site similarities and _ 
differences in an effort to answer the question. ' 

This qualitative analysis technique has a number of uses or strengths. 
For one, it can be used for hypothesis development and theory-building. When 
one. of the research teams has an idea, this is a ' "semi-structured" way to 
validate and flejh-out the idea. Hypotheses may then emerge which will lead 
to new approaches to analyzing the quantitative data collected as part of this 
study. The process of developing topical reports has already been used by 
CBAM project staff members as a way of collecting cross-site data and 
researcher insights for development of papers presented this conference. 

' Researcher Cross-Site Discussions . In addition to coordinating the 
structured data collection procedures at their sites, researchers were 
constantly involved in collecting ethnographic notes about impressions, 
hunches and possible hypotheses about principal behavior. Besides the 
researchers' informal notes, other qualitative sources of information about 
activities in the school 'districts were regulaprly tapped. These included 
subscribing to a local newspaper that covered the news within each of the 
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districts, and occasional contacts with^other disti^ict informants. These 
data sources were used by the researchers in a series of cross-site 
discussions. 

The study desigr^al so 'i nfluenced how researchers are being grouped for 
various cross-site discussions. The three districts in the study were 
selected based on whether their schools were in their first, second, or third 
year of impl emen^tation of a district-wide curriculum innovation as of 
September, 1980. In each of the three districts, three schools were selectek^ 
based on the leadership style of their principals. The principals were 
identified based on the expert judgment of district administrators that they 
were one of three hypothesized leadership styles: responder, manager, or . 
initiator (Hall, Rutherford & Griffin, 1982). One researcher was designated 
to work with each principal. ' ' ■ ^ 

One set o*f discussions has been' centered around the change facilitator 
style of the school principal. In the. study three researchers, each involved 
with a different district, were working with responder principals; three were ^ 
. working with manager principals, and three were working with initiator 
principals. By grouping researchers in this manner, it has been possible to 
. have interesting and meaningful discussions that focused not on any single 
distri'ct, \onovation, or year of implementation, but strictly on the similar 
and different characteristics of principals who share a common change 
facilitator' style. These discussions have proved very useful In the 
development of the behavioral profiles for each change facilitator style. 

By grouping researchers according, to the district in which theywere 
working, it has been possible to have focused discussions, about several 
topics, including the district, the ^iear of implementation, and the 
■innovation. • In one set of" discussions, the researchers are discussing 
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district influences that affected the change effort in the three schools being 
studied. Another set, of discussions by the same group is focusing on the year 
of implfeiT|entation and how being at this particular phase of the change process 
h?is affected the effort. Finally, this same group is having another sej: of 
d'iscussions related to the characteristics of the innovation'being implemented 
and how innovation-specific matters influenced the change process.- 

The researcher cr^s-site discussions have been a very useful tool in- 
allowing researchers to cross-check their impressions with others who are 
knowledgeable about the setting. The use of this process, has given the 
researchers increased confidence that the qualitative data base truly 
represents reality and tjT^t if can be used to verify and enhance the 
quantitative data in the -study. 

Site Representative/Researcher Discussions . Another procedure tl^at has 
been used in the analysis of the qualitative data is site-representative/ 



researcher discussions. In February, 19,82 of the data ' analysi s year, one 
representative from each of the three districts was brought to the Research 
and Development Center to wofk with the researchers' fQr two days. The 
representative were all school district central office personnel who' were 
knowledgeable about the innovation being studied and the district policies and 
procedures related to implementation of the' innovation. In addition, each 
site-representative is acquainted with, and has worked with, the .three 

t 

principals from their districts that were in the study. ■ . 

During the two-day work . session , in^itial findings from the .study and 
researcher impressions were shared with the site-representatives. The 
researchers used this opportunity to clarify quesitons they had about the data 
and to gain inpu^from the site-representative. Site-representatives were 
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asked to be very candid and to use their background knowledge to- test CBAM 
"findings and impressions. They were encouraged to give- both' positive and ' 
negative feedback, to ask tFTeir own questions, and to alert us when our 
impressions were not consistent with their understaja'ding of the context. 
Identified discrepancies between researcher and site-representative 

ft? 

impressions led to further discussions in which site-representatives were able 
to share their background knowledge and unders^tRding and provide; the • 
researchers with additional insight. . ■ ■ ■ 

• ■ The site representative/researcher discussions served several 

A. J • - 

constructive purpd'ses:' First of all, site representatives were able to react 

to researchers' impressions and thus verify or modify already existing data. 

Secondly, the site representatives were most helpful in providing additional 

information and insight which can be used by researches as they continue their 

work. Finally, the site representatives served the function of being a ^ 

, "reality check" for the data base. , . 

■ Conclusions 

The useof qualitative methods as a part of the research strategy and 
analysis in the PTI" study had the major advantage of allowing a chain of 
evidence to emerge that complemented and enlarged upon quantitative methods. 
^ The use of qual itative ^and, quantitative metho^s^ combination became the 
framework through which study questions were- approached in data analysis. In 
this early stage ^effanaTW si s , interaction between quantitative and qualitative 
methods has beenMimi^d to the coding of interventions- and the listing of 
effects as shown irf interviews, Levels of Use, Configuration and Concerns data 
(Figure 5). Further .work will involve frequencies of types of .interventions 
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as related to principal style and to year of implementation. . Beginning data 
analysis itself demanded a means to draw from researchers' own knowledge base 
■and experience at their sites in order to have a sufficient foundation from 
v^hich to draw conclusions. This is a p^ractical issue as well as a 
methodological one. The focused qualitative procedures used in the course of 
analysis -of data are one means for resolution of the problem of verification, 
cross reference, and. validation of the data as contained in quanti tat|^ve 
measures,. as well as a way to approach theory development. ' ' 

The p-rocedures designed to* focus qualitative information were geared to 
get certain kinds of information. The Intervention map allows for an overview 
of the principal's plan of action for change, as it was implemented in the • 
school site. The- Antecedent ,[nap shows how one change in the school' led to _ 
another. The Critical Incident map and time line lends a sense of context to 
the relationship of events, to each other indicating significant evenls which 
served as catalysts to action during the school year. The information 
contained in effects allowed for the derivation of links between interventions 
and- hypotheses formulation. They further served as a means to relate 
^qualitative informa.tion about specific interventions to the quantitative 
coding and 'data . collection measures. The Case Study Interv.iew focuses 
researcher perceptions as "site experts" to tl^e study questions. The Case 
Study Topical Reports uses "some of the same information as applied to specific 
topics across sites. Discussions 'between research staff allowed researchers 
to express their impressions and have "them cross-checked by^ other researchers 
who have had similar experiences. The Site Researcher/Pi str.i ct Representative 
Discussior^/ have a1 lowed site^ representatives to react to researcher 
impressions, provide additional information and insights, and serve as a 
"reality check'' for the data ba^se. . ' 



The use of 'convergent methods -in^/ata collection and data analysis/ is not 
a new idea.'^As mentioned ia earl ier jjj scussion , social scientists have 
referred ^to this as "triangul ation" or^"the combination of methodologies, in 
the study of the same phenomena" (Jick, 1979, p. 602). The intent, of 

4 ' ' ■ * 

triangulation is to allow for cross validation and verification of data 
through congruence of measures. As occurred in the PTI 'study, Jick suggests , 
'that researchers using qualitative methodology use "systematic observations, 
utilize satnpling techniques, and develo^p quanti f i,abl-e schem^^ for coding" 
(1979, p. 604)... The PTI study did this through the observations and 
interviews conducted at four jDoints in the year', through the use of SoC, LoU, 
and Innovation CoVif iguration data, and through the coding of interventions. 
The final integration arjd interpretation of results from these combined 
methods is as. yet in the proOcess of development i,n terms of this study. 
Possible further steps include development of a. means .to relate the the 
information contained in effects to the codification procedures used to 
describe interventions. One theoretical goal would be to include effects in 
the codification scheme. Other work will involve frequencies of certain 
interventions and what they mean, a descriptive taxonomy of principals' styles 
in facilitation, and an invespg^ion of other factors influencing principal 
efforts, such as bias or uncot^ciou?, actions. 

At this point in data anal^s^^ qi^jU^ative procedures used have allowed 
for sufficient depth of site overvieWHo begin more detailed work. The final 

i 

test of use of thfese measures will come ^n developing results through 
continuing analysis and applying those results to change theory. 
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